Combination of words and word categories in varigram histories

This paper presents a new kmd of language models: caregor@vord varigrums. This special model type permits a tight integration of word-based and category-based modeling of word sequences. Any succession of words and word categones may be employed to descnbe a given word history. This provides a much greater flexibtlity than previous combinations of word-based and category-based language models. Expenments on the WSJO corpus and the 1994 ARPA evaluation data indicate that the category/word vangram yields a perplexity reduction of up to 10 percent as compared to a word vangram of the same size. and improves the word error rate (WER) by 7 percent. Compared to a linear interpolation of a word-based and a category-based n-gram, the WER improvement is about 4 percent.